Learned-SBL-GAMP based hybrid precoders/combiners in millimeter wave massive MIMO systems

In Millimeter-Wave (mm-Wave) massive Multiple-Input Multiple-Output (MIMO) systems, hybrid precoders/combiners must be designed to improve antenna gain and reduce hardware complexity. Sparse Bayesian learning via Expectation Maximization (SBL-EM) algorithm is not practically feasible for high signal dimensions because estimating sparse signals and designing optimal hybrid precoders/combiners using SBL-EM still provide high computational complexity for higher signal dimensions. To overcome the issues of high computational complexity along with making it suitable for larger data sets, in this paper, we propose Learned-Sparse Bayesian Learning with Generalized Approximate Message Passing algorithm (L-SBL-GAMP) algorithm for designing optimal hybrid precoders/combiners suitable for mmWave Massive MIMO systems. The L-SBL-GAMP algorithm is an extension of the SBL-GAMP algorithm that incorporates a Deep Neural Network (DNN) to improve the system performance. Based on the nature of the training data, the L-SBL-GAMP can design the optimal Hybrid precoders/combiners, which enhances the spectral efficiency of mmWave massive MIMO systems. The proposed L-SBL-GAMP algorithm reduces the iterations, training overhead, and computational complexity compared to the SBL-EM algorithm. The simulation results unveil that the proposed L-SBL-GAMP provides higher achievable rates, better accuracy, and low computational complexity compared to the existing algorithm, such as Orthogonal Matching Pursuit (OMP), Simultaneous Orthogonal Matching Pursuit (SOMP), SBL-EM and SBL-GAMP for mmWave massive MIMO architectures.


Introduction
During the last few decades, many end users are connected to the internet due to the technological enhancements in communication devices [1,2].As a result, lower frequency bands have been congested.To tackle this issue, scientists have explored new ways, such as equipping base stations with massive antennas [3,4], spectrum coding, and network densification algorithms [5].These ways have solved the issues; however, they are not enough to meet the everincreasing demand of users.This situation has shifted the focus of research toward millimeterwave frequencies (mmWave) [6].
The spectrum of mmWave ranges from 30GHz-300GHz, and it is deemed a perfect candidate to meet the requirements of 5G networks, as they require higher spectral and energy efficiencies [7].However, the main issues associated with mmWave frequencies are related to high path losses and severe blockages [8].It has been seen that the large antenna array at the base station helps to significantly reduce high path losses [9].Meanwhile, a large antenna array can easily be installed at the base station owing to the small wavelength of mmWave frequencies.Therefore, the incorporation of mmWave with large antenna arrays at the base station is one of the most suitable options to fulfill the requirements of a current and future generation of communication networks.A large antenna array at the base station is another promising wireless technology to fulfill the requirements of spectral and energy efficiencies for the current and future wireless communication systems [10].
Contrary to the conventional MIMO systems [11] where each transceiver chain is equipped with the individual Digital to Analogue Convertor (DAC), Analogue to Digital Convertor (ADC), frequency matches, and amplifiers, researchers opt for hybrid architectures in mmWave communication systems to reduce the cost.In hybrid architecture, the precoders and combiners are designed by a combination of analog and digital precoders and combiners for processing the transmitting and receiving signals along with reducing the cost.We get high beamforming gain due to the analog domain and high multiplexing gain with reduced spectral efficiency due to the digital domain [12,13].Also, along with the cost reduction in mmWave systems, the complexity and the power efficiency of the system get improved because of using fewer RF chains compared to the antennas.
Furthermore, due to the above-mentioned propagation losses at the channel in mmWave systems, the response of the channel gets sparse because the propagation characteristics of mmWave are different compared to microwave systems [14,15].Also, due to the sparse channel in mmWave, the nature of the hybrid precoders gets sparse.Many attempts have been made in the literature to address the hybrid precoding problem using various algorithms [16].In [17,18], the authors propose a codebook-based hybrid precoding design by considering the sparse nature of the mmWave channel.Moreover, Orthogonal Matching Pursuit (OMP) is deemed a good candidate for designing hybrid precoders.However, with the increment in the number of transceivers, the computational complexity gets increased in OMP [19,20].As a result, it brings a poor estimation of the channel matrix due to the increment in the computational complexity, which reduces the spectral efficiency.Another approach, named as greedy Simultaneous Orthogonal Matching Pursuit (SOMP) technique, is used to reduce the system's computational complexity.Although the computation complexity in terms of the number of iterations gets reduced in SOMP, the algorithm still leads to poor performance when we increase the dimension of transceiver chains [21] which reduces the spectral efficiency.
In [22], the authors propose a greedy frequency selective hybrid precoding algorithm based on Gram-Schmidt orthogonalization to reduce the complexity during the designing of precoders/combiners.The Gram-Schmidt orthogonalization algorithm is based on the greedy selection of columns of the transceiver matrix by considering minimal spatially active channel components to reduce the approximation error.In [23], the authors propose a frequencyselective hybrid precoding based on maximizing mutual information with the help of quantized codebooks.However, similar to the issues in OMP, the proposed schemes lead to poor performances when the number of transceiver elements gets increased.
In [24], the authors propose a Sparse Bayesian Learning (SBL) estimation scheme for the evaluation of posterior functions to the prior information, improving the channel's estimation accuracy.Motivated by this research, the authors discuss the computation of sparse channels with the help of SBL estimation schemes [25][26][27].They developed a Single Measurement Vector (SMV) for SBL-based mmWave MIMO channel estimation and then moved to the development of Multiple Measurement Vectors (MMV) for multi-user communication.
In [28], the authors propose a novel SBL framework to estimate the spatial signatures and temporal varying characteristics of the sparse virtual channel model.To avoid the unacceptable complexity, the expectation maximization (EM) algorithm is applied to approximate the maximum likelihood (ML) solution iteratively.Furthermore, in [29], the authors estimate the channel of a massive MIMO system by using the EM-based Variational Bayesian (EM-VB).By exploiting the channel reciprocity between the uplink and downlink, the downlink channel is estimated.In terms of methods, the authors use the SBL via EM approach for the computation of a time-varying channel in both of these papers.
In terms of precoder designing, the SBL method is used to minimize the error between the desired and the estimated precoders.For the estimation of missing information, the Expectation Maximization (EM) based SBL method is used during the designing of hybrid precoders, as EM-based SBL assists to compute the posterior distribution of hidden variables [30,31].In addition, it has been seen that the SBL works well for signal recovery, and in fact, it shows better performance compared to greedy methods and other basis pursuit methods.Despite exhibiting good performance, the dimension of the signal gets increased because we have to compute inverse elements having NxN size.
In [32], the authors propose a Generalized Approximate Message Passing (GAMP) algorithm for the estimation of the mmWave channel in a computationally efficient way.The integration of approximate message passing with EM assists to compute the distribution of true posterior approximately.Moreover, it has been seen that the performance of SBL with EM and GAMP is the same in terms of channel estimation accuracy, but the complexity is less in the latter one [33].In this paper, we work on the design of hybrid precoders/combiners for mmWave massive MIMO hybrid architecture.The main purpose of using the hybrid architecture is to reduce the cost along with the power consumption, as RF chains are less than the transmitting antennas in the hybrid architecture.The nature of the precoders gets sparse in mmWave systems, we have proposed a deep learning-based architecture for designing the optimal hybrid precoders/combiners by getting motivated by the success of SBL via the GAMP algorithm for the estimation of mmWave channels in [34][35][36].Firstly, the SBL via the GAMP algorithm is used for designing optimal hybrid precoders/combiners for mmWave massive MIMO systems in a computationally efficient way.To further improve the estimation accuracy and the channel overhead, we have proposed a deep learning-based approach, named Learned-SBL-GAMP (L-SBL-GAMP) algorithm, to design the hybrid precoders/combiners in mmWave massive MIMO systems.Actually, in the L-SBL-GAMP algorithm, all T layers are used to estimate the hybrid precoders/combiners and its matrices are sparse.The simulation results unveil the effectivities of the proposed deep learning-based method, where it can be seen that the proposed deep learning-based algorithm outperforms the existing methods (LS, OMP, SOMP, SBL-EM, and SBL-GAMP) by providing the reduced Normalized Mean Square Error (NMSE) value and the increased spectral efficiency.
In summary, Table 1 provides the pros/cons of the recent state-of-the-art work in the literature.
The main contributions and novelties of the paper are summarized as follows: 1.In this paper, we provide the mathematical modeling of optimal precoders/combiners for mmWave massive MIMO hybrid architecture.
2. To design hybrid precoders/combiners, we utilize the L-SBL via the GAMP algorithm, which reduces the computational complexity of the system compared to the EM-based SBL approach.
3. Due to the fixed nature of hyperparameters during the iterations in SBL via GAMP, the estimation accuracy of precoders gets reduced.Therefore, to tackle this issue, we propose a deep learning-based approach named learned SBL via GAMP.
4. In the proposed methodology, the hyperparameters get effectively tuned during each iteration, which assists in achieving the minimum loss function.The proposed DNN architecture can design the optimal hybrid precoders/combiners. 5. To validate the effectiveness of the proposed methodology, the experimental results are provided where it can be seen that the proposed methodology outperforms the existing approaches (LS, OMP, SOMP, SBL-EM, and SBL-GAMP) by providing less Normalized Mean Square Error (NMSE), losses, and the enhanced spectral efficiency.

Notations
we opt for the following notations in this paper: A (.j) and A (i,j) represent the i th row, j th column, and (i, j) th element of a matrix A, respectively.[�]* represents a conjugate operator, [�] H represents a conjugate transpose operator, and [�] T represents a transpose operator.a is a vector and a is a scalar.E(�) represents an expectation operator, kXk F represents a Frobenius norm of X, and denotes the l 2 norm of X.

System model
The architecture of hybrid precoders and combiners for the mmWave massive MIMO system is depicted in Fig 1 .Let N t be the transmitting elements at the transmission section, and several receiving antennas at the reception side.N RF t represents the RF chains at the transmission, and represents the RF chains at the receiver side.It also N S represents the number of data streams transmitted via RF chains at the transmitter and receiver sides.F BB � N RF t � N S denotes the baseband precoder, and F RF � N t � N RF r is the RF precoder at the transmitter side.W BB � N RF r � N S denotes the baseband combiner, and W RF � N r � N RF r represents the RF combiner.Furthermore, we impose total power constraints to normalize the baseband precoder [31].The numerical model for the channel with L paths is represented in Eq (1), which shows the channel correlation between the transmitting and receiving antennas.The geometric channel model is expressed as H�N r ×N t , and it is written as: Where represents the complex gain for the lth path.φ r,l ,θ r,l denote the Angles of Arrivals (AoAs) received for the lth path.Denote the Angles of Departures (AoDs) at the transmitting antenna for the lth path.a t (�) denotes the transmit array response and a r (�) denotes the receive array response.Due to mmWave frequencies, there are a small number of available channel paths L.

Problem definition
This section deals with the designing of hybrid precoders/combiners that optimally increase the system's achievable rate while suppressing channel training overhead.For the given system and channel models, the achievable rate can be written as [37]: Where F = F RF F BB denotes the analog/digital precoder, and denotes the analog/digital combiners.The covariance matrix can be written as: The signal-to-noise ratio (SNR) of the system can be written as: Where represents the total transmit power, and denotes the noise power.The designing of optimal hybrid precoders and combiners is required to improvise mutual information between the baseband precoder and RF precoder, which can be written as [31]: The direct maximization process is difficult; therefore, an ideal digital precoder and combiner method is used.The ideal digital precoder and combiner can be written as: Where denotes the right singular vector matrices, and U 2 C N r �N r denotes the left singular vector matrices.The values of matrices are estimated from the Singular Value Decomposition (SVD) H.
The problem of maximizing the mutual information concerning the precoders F BB and F RF can be well approximated by the sparse matrix reconstruction problem, and it is formulated as below: The equivalent precoders and combiners design problem for the mmWave MIMO system to achieve the best approximation to the ideal precoder F opt and ideal combiner W opt is shown below in Eqs ( 9) and (11) [31]: Where A T ≜½a t ðy t;l Þ; a t ðy t;2 Þ; . . .:; a t ðy t;G Þ� 2 C N t �G denotes the dictionary matrix of transmit array response, and A R ≜½a r ðy r;l Þ; a t ðy r;2 Þ; . . .:; a r ðy r;G Þ� 2 C N r �G denotes the dictionary matrix of receive array response.
The first constraint arises because Fopt BB 2 C G�N S can have only N RF t � K nonzero rows corresponding to the number of active RF chains, while the second constraint kA T FBB k ¼ N S .

Sparse Bayesian learning (SBL) algorithm
In this section, a two-layer hierarchical prior model is proposed to calculate the sparsity of a Bayesian learning model.The first layer represents the Gaussian prior distribution [32]: Where denotes the non-negative hyperparameter, which controls the sparsity coefficient F BB n .In the next layer, hyperpriors are computed with the help of Gamma distributions: The sparse signal and the hyperparameters are learned using the Expectation-Maximization (EM) algorithm.In the EM algorithm, F BB is a hidden variable.Also, using the iterative method, we strengthen a lower bound value on probability p(α,γ|F opt ).The EM algorithm works in two steps named E-step and M-Step, respectively.Firstly, the F BB is computed, and it is given as: pðF BB jF opt ; α ðtÞ ; g ðtÞ Þ / pðF BB jα ðtÞ ÞpðF opt jF BB ; g ðtÞ Þ ð17Þ From the posterior function pðF BB jF opt ; α ðtÞ ; g ðtÞ Þ, the Gaussian distribution of its mean and covariance matrix is written as: The posterior pðF BB jF opt ; α ðtÞ ; g ðtÞ Þ obeys the Gaussian distribution, and its parameters are: The Q-function is maximized in M-steps by keeping {α,γ}, which leads to the following update rules: Where is the expectation concerning?It is observed that the EM algorithm for every iteration has to update the posterior distribution with G×G computation inverse.Hence, the EMbased sparse Bayesian has complexity in the range of O(G 3 ) flops, which increases further concerning larger dimensions [32].
The optimal hybrid precoder and combiner are designed with the help of SBL based EM method.The SBL-EM method achieves high spectral efficiency and it increases the training overhead with computational complexity.To minimize the number of iterations and complexity, we use the SBL via the GAMP algorithm.The SBL via the GAMP algorithm does not require an inverse operation.It only needs a simple multiplication operation.

Proposed learned SBL-GAMP architecture
In this section, we first discuss the SBL via the GAMP algorithm and then move to the proposed L-SBL-GAMP algorithm.

SBL-GAMP algorithm
The GAMP algorithm attains the target with a smaller number of iterations and training overhead [32].The Bayesian iterative technique, named GAMP, is adapted to estimate the optimal hybrid precoder of mmWave systems.The proposed algorithm provides low computational complexity and finds the estimation of a posterior distribution conditioned on the prior distribution F BB .In the EM algorithm, the posterior estimate distribution F BB is used to swap the true posterior distribution.Moreover, the hyperparameters are considered as known and fixed in the GAMP algorithm.Depending on the estimated values F BB , its parametric values are updated in the M-step.
In the GAMP algorithm, we use the central-limit-theorem to approximate variable and factor nodes, which enhances the performance of the system.The SBL-GAMP framework adopts two important approximation methods.In the first approximation method, the GAMP assumes posterior independence among hidden variables {F BBn }, which θ = {α,γ} denotes the hyperparameters.
The true posterior distribution approximates as p(F BBn |F opt ,θ), and it is computed as: Let rn and are different variables that are updated for every process value of the GAMP algorithm.With the help of Gaussian Prior Distribution, the approximate posterior pðF BBn jF opt ; rn ; t r n ; θÞ of mean and variance is given in Eqs ( 26) and (27).
Where pm and t p m are the parameters that are iteratively updated during the iterative process of GAMP.
Using the Gaussian prior distribution pðF m jF opt m ; pm ; t p m ; θÞ defines the mean and variance, and it is given in Eqs ( 29) and (30).
With the above two approximation methods, the two following important scalar functions are defined as gin(�) and gout(�), which are utilized for the proposed approach.gin(�) denotes the scalar function of input, which defines the posterior mean m F BB n , and it is expressed by using Minimum Mean Squared Error (MMSE) mode [34]: To obtain the estimated variance � F BB n , we take the partial derivative t r n ginðr n ; t r n ; θÞ with respect to t r n : Moreover, gout(�) denotes the output scalar function, and its posterior mean m F m is defined as: Taking the partial derivative of goutðp m ; t p m ; θÞ to obtain the posterior variance � F m : The GAMP algorithm finds the posterior mean m F BB n ðkÞ and variance F BBn at the k iteration for the given input gin(�) and output gout(�) scalar functions.Finally, the approximate posterior distributions for the variables F BB F m = A T F BB are derived by using the proposed GAMP algorithm.The GAMP algorithm does not require an inverse operation.It only needs a simple multiplication operation, which can be scaled as O(N t G).Thus, its computational complexity gets considerably reduced.
As far as learning hyperparameters is concerned, the EM method is used to estimate hyperparameters by considering hidden variables F BB and the iteratively improving Q-function: To find the number of M-steps for the analysis of hyperparameters {α n }, we take the firstorder derivative of the Q-function for α n : @ @a n Qðθjθ ðtÞ Þ ¼ @ @a n E F BB jF opt ; θðtÞ ½logpðθjF BB ; F opt Þ� ð36Þ ¼ @ @a n E F BB jF opt ; θðtÞ ½logpðθjF BB ; F opt Þpða n ; a; bÞ� ð37Þ Where represents the expectation operator regarding p(F BB |F opt ,θ (t) ).The approximate posterior distribution F BBn is used due to the unavailability of a true posterior.The GAMP algorithm is used to approximate the posterior distribution by replacing the true posterior distribution.From the Gaussian distribution process, the approximate posterior distribution pðF BBn jF opt ; rn ðk 0 Þ; t r n ðk 0 Þ; θ ðtÞ Þ defines the mean and variance, which are given in Eqs ( 26) and ( 27) Here, hyperparameters are required to be updated.The function of a hyperparameter is to inverse noise variance.The approximate posterior distribution of output without any disturbance is considered in the GAMP approach.The F is then assigned to the hidden variables for understanding noise variance, and it is given as: FjF opt ; yðtÞ ½logpðF opt jF; gÞpðg; c; dÞ� ð41Þ From the partial derivative of Q-function regarding γ projects: @ @g E FjF opt ; θðtÞ ½logpðF opt jF BB ; gÞpðg; c; dÞ� ð42Þ h�i specifies the expectation for pðF m jy; pm ðk 0 Þ; t p m ðk 0 Þ; θ ðtÞ Þ, and the approximate posterior distribution F m .We know that F m tracks the Gaussian distribution in terms of mean and variance, and it is expressed as: where m F m and � F m from ( 29)- (30) with fp m ; t p m g relocated with fp m ðk 0 Þ; t p m ðk 0 Þg and γ is changed with γ (t) .By equating the derivative to zero, the updated rule for: As of now, the GAMP-based sparse Bayesian learning algorithm is derived.For a better understanding, we have summarized the SBL-GAMP approach in algorithm 1 [32].
Based on derivates of the GAMP algorithm, hyperparameters are updated according to (40) and (45) using the EM method.
However, in SBL via GAMP, the hyperparameter is considered fixed for all the iterations, which reduces the estimation accuracy of precoders.To overcome this issue, the learned SBL via GAMP is proposed.In the learned SBL via GAMP, for each layer, the hyperparameters get changed and tuned to obtain the minimum loss function.

Learned SBL-GAMP architecture
The architecture of L-SBL-GAMP with multiple layers is shown in Fig 2 .Each layer of the L-SBL-GAMP is similar to an iteration of the SBL-GAMP algorithm.The inputs of the L-SBL-GAMP network are the F opt (Ideal digital Precoder) and A T (transmit array response).The outputs of the L-SBL-GAMP network are baseband precoder and trainable parameters θ = {W} = {α,γ}.In algorithm 2, for the first layer, fix them to learn W 1 = {α 1 ,γ 1 } and re-learn W 1 .In layer 2, fix the to learn them W 2 = {α 2 ,γ 2 } and re-learn W 0 , W 1 and W 2 .Similarly, the process will continue till the T layer, and it is shown in algorithm 2.

Algorithm 2: Learned SBL-GAMP
to minimize the loss function First estimate optimal precoder Fi BB ¼ GðF i opt ; F i RF ; θÞ and then find the loss function

End for
for R steps do Re-Learn θ t ¼ fW l g t l¼1 to minimize the loss function Estimate optimal precoder Fi BB ¼ GðF i opt ; A i T ; θÞ and then calculate the loss function

End for
Output: θ T−1 Algorithm 2, θ = {W} = {α,γ} denotes the set of learnable parameters in the t-th layer.The total number of L-SBL-GAMP layers is denoted as T, and R denotes the total number of minibatches used in the training of the L-SBL-GAMP layer.The output vector F i opt is related to the sparse baseband precoder vector F i BB : The loss function used in the training of the L-SBL-GAMP layer is the mean square error between the true sparse vector and the current training layer output.The expression for the mean square error loss function is given by: where m denotes the number of training samples in a mini-batch and G represents the function learned by L-SBL-GAMP.A layer-by-layer method is used, and the Adam optimizer is used in the back-propagation process to optimize the final value.The L-SBL-GAMP network is implemented in Python using the network libraries Keras and TensorFlow.
After estimating the sparse precoder FBB 2 C G�N S , we are following a two-step procedure to extract the baseband precoder F BB 2 C N RF t �N S .We calculate the transmit baseband precoder matrix from FBB .Firstly, we separate the G − K rows FBB to get FBB 2 C K�N S .The rows of the matrix are then removed to finally obtain the matrix.The transmit RF precoder F RF can be extracted by keeping the N RF t columns corresponding to the rows extracted from FBB .The above same procedure is utilized to calculate the baseband combiner ŴBB and W RF RF combiner.They can be extracted from by retaining the N RF r columns corresponding to the rows extracted from ŴBB .
Moreover, and is the optimum MMSE combiner.The estimated baseband precoder, RF precoder, baseband combiner, and RF combiner are substituted in Eq (2) to obtain the achievable rate or Spectral Efficiency [37].
Finally, the Normalized Mean Square Error (NMSE) is used to evaluate the performance of the L-SBL-GAMP network:

Computational complexity and performance analysis
Sparse Bayesian Learning (SBL) is a popular method for sparse signal recovery or optimal hybrid precoder design.SBL-EM (Expectation-Maximization) and SBL-GAMP (Gaussian Approximation Message Passing) are two variants of SBL that differ in their algorithms for estimating the sparse signal or designing the hybrid precoder.
In terms of performance, the accuracy of the optimal hybrid precoder design using SBL-EM is almost the same compared to SBL-GAMP, but at the cost of higher computational complexity.
The L-SBL-GAMP algorithm is an extension of the SBL-GAMP algorithm that incorporates a deep neural network to improve performance.The L-SBL-GAMP approach can achieve high accuracy while reducing the computational complexity compared to SBL-EM.However, the performance of the L-SBL-GAMP approach may depend on the quality and quantity of training data used to train the neural network.
Overall, while the L-GAMP-SBL algorithm has a higher computational complexity than GAMP-SBL and reduces computational complexity than SBL-EM, the improved performance it provides may make it the best choice in many practical applications where accuracy is critical.The choice of algorithm ultimately depends on the specific requirements of the application and the trade-off between computational complexity and performance.
SBL-EM has a computational complexity of O(G 3 ), where G is the number of grid matrices.The SBL-GAMP algorithm does not require an inverse operation.It only needs a simple multiplication operation, which can be scaled as O(N t G).Thus, its computational gets considerably reduced.
The computational complexity of L-SBL-GAMP is dominated by the training process, which typically has a complexity of O(ETN t G), where E is the number of epochs, T is the number of training samples or layers, N t is the number of transmit antenna and G is the number of grid matrix or dictionary matrix.
In general, the computational complexity of L-SBL-GAMP is higher than that of SBL-GAMP, due to the added complexity of the training process.However, this added complexity may be justified if the improved accuracy of L-SBL-GAMP leads to better performance in practice.It is also worth noting that there are techniques for reducing the computational complexity of the training process, such as using mini-batch training or pre-training the neural network on a smaller dataset before fine-tuning it on the full dataset.
The L-SBL-GAMP algorithm has a higher computational complexity than SBL-GAMP, as it involves additional processing steps required to train and use the deep neural network.However, the computational complexity of L-SBL-GAMP is still significantly lower than that of SBL-EM.
Overall, while the L-SBL-GAMP algorithm has a higher computational complexity than SBL-GAMP, it is still significantly more efficient than SBL-EM, and the improved performance it provides may make it the best choice in many practical applications.higher computational complexity than GAMP-SBL and lower computational complexity than the SBL-EM method.The choice of algorithm ultimately depends on the specific requirements of the application and the trade-off between computational complexity and performance.
The same parameters are used for both Figs 3 and 4. The spectral efficiency against SNR for the proposed L-SBL-GAMP and existing algorithms is shown in Fig 4 .The proposed L-SBL-GAMP algorithm achieves better spectral efficiency with a small number of iterations compared to the other existing algorithms.The SBL-GAMP algorithm and SBL-EM method achieve approximately the same achievable rates, but the computational complexity is higher in the SBL-EM method compared to the SBL-GAMP and proposed L-SBL-GAMP algorithm.
Fig 5 shows the NMSE against the number of paths for the proposed algorithm.The path element L varies from 2 to 10.The large number of paths gives a high computational cost with high estimation error.This is due to the small neighboring angular distance.This leads to the uncertainty of adjacent points, which affects the estimation of hybrid precoder performance in mmWave systems.However, the proposed L-SBL-GAMP algorithm shows better performance compared to other existing algorithms.
The performance of the number of paths against the spectral efficiency with the proposed method is unveiled in Fig 6 .The increased channel path decreases the spectral efficiency of the proposed algorithm.Moreover, increasing sparsity level leads to high errors in the precoder along with the high computational complexity in the mmWave channel.
In     error (NMSE) for varying numbers of RF chains.The results indicate that the proposed L-SBL-GAMP network achieves high spectral efficiency and superior accuracy as the number of RF chains increases.This improvement in spectral efficiency is due to the use of a larger set of random precoders, resulting in a decrease in estimation error through compressed sensing and an increase in the number of effective combining and precoding beam patterns.Consequently, it is preferable to use a larger number of RF chains to attain high spectral efficiency and reduce estimation error in mmWave systems.However, the use of a large number of RF chains increases the complexity of the mmWave architecture.Therefore, a trade-off between complexity and performance must be considered for real-time implementations.
The curves depicting the normalized mean squared error (NMSE) versus the number of antennas (N) for the proposed and existing approaches are presented in Fig 10 .As N increases, the NMSE value decreases for both the proposed and existing algorithms.However, the least squares (LS) algorithm shows almost constant performance for different N values.The proposed L-SBL-GAMP network demonstrates superior performance compared to other approaches.Although the proposed method achieves better results with an increased number of antennas, there is a corresponding increase in computational complexity.

Conclusion
In this paper, a deep learning architecture named L-SBL-GAMP is proposed to design optimal hybrid precoders/combiner for the mmWave massive MIMO hybrid architecture.The

Fig 7 ,
the Layer vs loss function is discussed for L-SBL-EM and the proposed L-SBL-GAMP algorithm.The proposed L-SBL-GAMP gives a reduction in the loss function compared to the L-SBL-EM network.Figs 8 and 9 display the performance of the L-SBL-GAMP network concerning signal-tonoise ratio (SNR) versus spectral efficiency (SE) and SNR versus normalized mean squared

Fig 8 .Fig 9 .
Fig 8. SNR Vs spectral efficiency (bits/s/Hz) for various RF chain.https://doi.org/10.1371/journal.pone.0289868.g008 Fig 11 displays  the number of antennas (N) versus the spectral efficiency for the proposed L-SBL-GAMP network and the existing method.the performance of the proposed L-SBL-GAMP network provides better spectral efficiency with an increase in the number of antennas for all hybrid systems.

Table 1 . A brief comparison of the pros/cons of the recent state-of-the-art work in the literature.
https://doi.org/10.1371/journal.pone.0289868.t001 BB denotes the noiseless output of the other approximation method, where a T m specifies the m th row in A T .The GAMP equates to the exact marginal posterior p(F m |F optm ,θ): pðF m jF opt ; pm ; t p m ; θÞ ¼ pðF optm jF m ; θÞNðF m jp m Let ε be pre-specified error tolerance.